<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<!--

Copyright (c) 2006, Sun Microsystems, Inc.
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

    * Redistributions of source code must retain the above copyright notice,
      this list of conditions and the following disclaimer.
    * Redistributions in binary form must reproduce the above copyright
      notice, this list of conditions and the following disclaimer in the
      documentation and/or other materials provided with the distribution.
    * Neither the name of the Sun Microsystems, Inc. nor the names of its
      contributors may be used to endorse or promote products derived from
      this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
THE POSSIBILITY OF SUCH DAMAGE.

-->
<head>
	<link href="/styles.css" media="screen" rel="stylesheet" type="text/css" /> 
 <title>JavaCC: CharStream Classes MiniTutorial</title>
<!-- Changed by: Michael Van De Vanter, 14-Jan-2003 -->
</head>
<body bgcolor="#FFFFFF" >

<h1>JavaCC [tm]: CharStream Classes MiniTutorial</h1>
<p>
This document describes in some detail the methods of the CharStream
classes. Note that some of the details may not be relevant for the
CharStream interface (to be used with USER_CHAR_STREAM).
</p>
<p>
There are 4 different kinds of char stream classes that are generated
based on combinations of various options.
</p>
<ul>

  <li> <h3>ASCII_CharStream</h3>

       Generated when neither of the two options - <tt>UNICODE_INPUT</tt>
       or <tt>JAVA_UNICODE_ESCAPE</tt> is set.

<p>

       This class treats the input as a stream of 1-byte (ISO-LATIN1)
       characters. Note that this class can also be used to parse
       binary files. It just reads a byte and returns it as a 16 bit
       quantity to the lexical analyzer. So any character returned by
       this class will be in the range <tt>'\u0000'-'\u00ff'</tt>.

</p>
  </li>
  <li> <h3>ASCII_UCodeESC_CharStream</h3>

       Generated when the option <tt>JAVA_UNICODE_ESCAPE</tt> is set
       and the <tt>UNICODE_INPUT</tt> option is not set.

<p>

       This class treats the input as a stream of 1-byte characters.
       However, the special escape sequence
</p>
<p>
             <tt>("\\\\")* "\\" ("u")+</tt> - (<i>odd number of backslahes followed
       by one or more 'u's.</i>)
</p>
<p>
       is treated as a tag indicating that the next 4 bytes following
       the tag will be hexadecimal digits forming a 4-digit hex number
       whose value will be treated as the value of the character at the
       position indicated by the first backslash. Note that this value
       can be anything in the range <tt>0x0-0xffff</tt>.
</p>

  </li>
  <li> <h3>UCode_CharStream</h3>

       Generated when the option <tt>UNICODE_INPUT</tt> is set and
       the option <tt>JAVA_UNICODE_ESCAPE</tt> is not set.

<p>

       This class treats the input as a stream of 2-byte characters. So
       it reads 2 bytes <tt>b1</tt> and <tt>b2</tt> and returns them as
       a single character using the expression <tt> b1 &lt;&lt; 8 | b2 </tt>
       assuming bigendian order. So in particular all the characters in
       the range <tt>0x00-0xff</tt> are assumed to be stored as 2 bytes
       with the first (higher-order) byte being 0.

</p>
  </li>
  <li> <h3>UCode_UCodeESC_CharStream</h3>

       Generated when both the options <tt>UNICODE_INPUT</tt> and
       <tt>JAVA_UNICODE_ESCAPE</tt> are set.
<p>

       This class input is a stream of 2-byte characters (just
       like the UCode_CharStream class) and the special escape sequence
</p>
<p>
       <tt>("\\\\")* "\\" ("u")+</tt> - (<i>odd number of backslahes followed
       by one or more 'u's.</i>)
</p>
<p>
       is treated as a tag indicating that the next 4 2-byte characters
       following the tag will be hexadecimal digits forming a 4-digit hex
       number whose value will be treated as the value of the character at the
       position indicated by the first backslash. Note that this value
       can be any value in the range <tt>0x0-0xffff</tt>. Also note that
       the backslash(es) and u(s) are all assumed to be given as 2-byte
       characters (with the higher order byte value being 0).

</p>
  </li>
</ul>

<h4>
Note : None of the above classes can be used to read characters in a
       mixed mode, i.e., some characters given as 1-byte characters and others
       as 2-byte characters. To do this, you need to set USER_CHAR_STREAM
       option to true and define your own char stream.
</h4>

<hr />
<p>
(Throughout the following, we use the notation XXXCharStream that stands
 for any of the above described 4 classes.)
</p>
<h3>Constructors</h3>
<ul>

  <li><tt>
  public XXXCharStream(java.io.InputStream dstream,
                                        int startline, int startcolumn)
</tt>

<p>
  Takes an input stream, starting line and column numbers and constructs a
  CharStream object. It also creates buffers of initial size 4K for buffering the
  characters and also for line and column numbers for each of those characters.

</p>
  </li>

  <li><tt> public XXXCharStream(java.io.InputStream dstream,
                                        int startline, int startcolumn, int buffersize)
</tt>

<p>
  Takes an input stream, starting line and column numbers and constructs a
  CharStream object. It also creates buffers of initial size <tt>buffsize</tt> for buffering the
  characters and also for line and column numbers for each of those characters.
</p>
<p>
  So when you have an estimate on the maximum size of any token that can occur,
  you can use that size to optimize the buffer sizes. Note, however, that
  these sizes are only initial sizes and they will be expanded as and when
  needed (in 2K steps).
</p>
 </li>
</ul>

<h3>Methods</h3>
<p>
All the following methods will be static or nonstatic depending on
whether the STATIC option is true or false at the generation time. Also only
those methods that users can use in their lexical actions (using the
<tt>input_stream</tt> variable of the lexical analyzer) are documented
here. Rest of the (public) methods are very tightly coupled with the
implementation of the lexical analyzer and thus <b> should not </b> be
used in lexical actions. In the future when we adopt version 1.1 of the Java [tm] programming language, we will
streamline this by making that part of the interface an innerclass to
the lexical analyzer.
</p>
<ul>

  <li> <tt>public final char readChar() throws java.io.IOException</tt>
<p>

       This method returns the next "character" in the input according
       to the rules of the CharStream class as described above. It will
       throw <tt>java.io.IOException</tt> if it reaches EOF during the
       process of "constructing" the character. It also updates the line
       and column number and buffers the character for any possible
       backtracking that may be required later. It also stores the line
       and column numbers for the same purpose.
</p>
  </li>
  <li> <tt> public final int getBeginLine() </tt>

<p>
       This method returns the line number for the beginning of the
       current match.
</p>
 </li>
 <li> <tt> public final int getBeginColumn() </tt>

<p>
       This method returns the column number for the beginning of the
       current match.
</p>
  </li>
  <li> <tt> public final int getEndLine() </tt>

<p>
       This method returns the line number for the ending of the
       current match.

</p>
  </li>
  <li> <tt> public final int getEndColumn() </tt>

<p>
       This method returns the column number for the ending of the
       current match.
</p>
  </li>
  <li> <tt> public final void backup(int amount) </tt>

<p>
       This method puts back <tt>amount</tt> number of characters
       into the stream. Note that the amount indicates the number
       of characters as constructed by <tt>readChar</tt>. Since the
       buffers used are circular buffers, it cannot check for
       illegal <tt>amount</tt> values, it just wraps around. So it
       is the user's responsibility to make sure that those many
       characters are really produced before a call to this method.
</p>
  </li>
  <li><tt> public final String GetImage()</tt>

<p>
       Returns the image of the current match. As far as the XXXCharStream
       is concerned, all characters after the last call to the private method
       <tt>BeginToken</tt> are considered a part of the current match.
</p>
  </li>
  <li><tt> public void ReInit(java.io.InputStream dstream,
                                        int startline, int startcolumn)</tt>

<p>

       This method reinitializes the XXXCharStream classes with a (possibly
       new) input stream and starting line and column numbers.
</p>
  </li>
  <li><tt> public void ReInit(java.io.InputStream dstream,
                 int startline, int startcolumn, int buffersize)</tt>

<p>

       This method reinitializes the XXXCharStream classes with a (possibly
       new) input stream and starting line and column numbers and adjusts
       the size of the buffers to <tt>buffersize</tt>, by extending them.
       Note that if the value of <tt>buffersize</tt> is less than the current
       buffer sizes, they remain unchanged.
</p>
   </li>
   <li><tt> public void adjustBeginLineColumn(int newLine, int newCol)</tt>
<p>

       This method adjusts the line and column number of the beginning of
       the current match to <tt>newLine</tt> and <tt>newCol</tt> and
       also adjusts the line and column numbers for all the characters
       in the lookahead buffer.
</p>
   </li>
</ul>

</body>
</html>
